Digital pathology provides new ways to visualize tissue slides and enables new workflows for analyzing these slides.
As illustrated in FIG. 1, a typical system 1 for digital pathology may notably comprise a scanner 10 for scanning a sample, for example a tissue sample and create a digital image thereof. Many types of scanner have already been proposed, such as 1D line scanner, 2D array scanner, so called Time Delay Integration (TDI) scanner, etc.
The system may further comprise a PACS 20 (Picture Archiving communication System) for archiving and communicating information such as a digital image of the tissue sample. The system may further comprise a computing device 30 for executing computer implemented instructions, and memory 40 such as buffer memory for loading such instructions and other binary information such as imagery data. The computing device may be a processor, FPGA (Fully Programmable Gate Array), DSP (Digital Signal Processor) and the memory can notably be a DRAM, SDRAM, SRAM, DDR SDRAM, FRAM, ROM, CD, DVD, depending on its intended function in the system and the desired performance.
As well known in the field of digital pathology, the computing device may generally be used for image processing, and image analysis for example for diagnosis. In the field of digital pathology, image processing generally refers to processing a digital image with an algorithm in order to modify the content of this image. Image processing thus notably comprises the functions of enhancing the perceived quality of the image, and registering the image in an image to a reference image. To the contrary, an image analysis algorithm (e.g. for diagnosis) will not modify the content of the digital image under study. In fact, this image may typically be one on which an image processing algorithm has already been applied. For information, a well-known example of an image analysis algorithm for diagnosis corresponds to a CAD algorithm (Computer Aided Diagnosis).
The present invention specifically pertains to the technical field in digital pathology of image processing as defined above. In other words, the present invention pertains to a method and software in digital pathology that prepare a digital image to be viewed and/or subsequently analyzed with an image analysis algorithm, like a CAD algorithm for example.
As mentioned before, an image processing algorithm may be configured to enhance the contrast of an image so that the operator can better find features of interest in the image displayed on a screen 50 of system 1. By way of example, the operator may first load an image for display on the screen showing a tissue sample stained with an H&E technique (Haematoxylin and Eosin). This image, which will be called herein after an HE image, may show the entire area of the sample that has been acquired with the scanner or a portion thereof. The operator may then select a region of interest, called herein after a sub-region, in this image in order to display a particular area of the sample at a higher resolution for example.
In this effect, system 1 may comprise an image processing algorithm which first transforms the entire area of the digital image, thereby creating a high resolution digital image of the entire area of the sample. Then, the further processing retrieves the imagery content in said sub-region from this high resolution image and the imagery content is displayed on the screen.
A problem however with such image processing algorithms is that, usually in digital pathology, the size of digital images are far bigger than that of digital images taken from other usual medical imaging instruments such as ultrasound, MRI, CT, PET, or X-ray.
As a consequence, in digital pathology, the large size of the digital images often require much more intensive computation and often lead to a low rendering performance notably in terms of display speed.
To solve such a problem, it has been proposed to optimize the hardware architectures of the computing devices by using for example dedicated graphics processing units (GPU) or parallel architectures. It has also been proposed to optimize the image processing algorithms. However, it is still difficult to achieve good performance with reasonable optimizations. Among others, such solutions again require a large amount of storage space and large bandwidth consumption notably between the storage in PACS and the computation device.
A different solution to the above mentioned problem has been proposed by down sampling (decreasing the resolution) the digital image representing the entire area of the sample and applying the transformation (e.g. enhancing the contrast) to this new image. When the user then selects the sub-region, the system retrieves the content of this region in the down sampled image. Even though this solution may provide real time performance, the down sampled image displayed to the pathologist may not have sufficient quality.
As mentioned before, image processing also refers to image registration and here again the above-described problems may occur.
As another typical example, a pathologist may desire to view an HE image and indicate relevant sub-regions (e.g. a tumour area). She may then desire to order at least one additional slide prepared with a different staining technique for further analysis, for example an immunohistochemistry (IHC) technique. The additional slide may be imaged with a scanner to generate an IHC digital image. The pathologist may then desire to view the HE and IHC images side by side on the screen.
However, due to the nature of the slide preparation and due to the digitization process, the tissue and its features may not have the exact same shape, appearance or spatial alignment, making it difficult to find the same region on different slides of adjacent tissue.
To make this possible, an image registration may be needed. It should be noted here that image registration designates herein a process of computing a spatial transformation that maps point from one image to homologous points in another image.
Image registration techniques are known in digital pathology and are usually based on applying such a transformation on each acquired digital image of the sample.
However, it may be reminded that digital microscopy images are usually different than digital radiology images. The size of the digital microscopy images is one of the differences that has been described before.
As another difference, radiology images with the exception of magnetic resonance (MR) images, tend to generate absolute pixel intensities (for example general X-ray and CT images use the Hounsfield unit where air is −1000, water is 0, and bone >400). Similar to MR, the pixel intensity distribution of digital microscopy images depends on a large variety of factors, which are often difficult to control and possibly not uniform for the entirety of the image or between scans. Such factors include tissue movement during microtoming, non-standardized staining procedures, and auto-focus and other image processing algorithms used during acquisition.
As another difference, the acquisition of multi-modal images within radiology typically involves imaging the same region (for example CT-PET acquisition captures the same anatomic region at the same time). Digital microscopy imaging generates multiple modes or representations by applying different staining procedures—such as immunohistochemistry or in-situ hybridization (ISH)—on adjacent tissue cross-sections. This form of acquisition is most similar (though still different) to intra-patient registration of radiology images.
The three differences explained above (i.e. extreme image size, non-absolute pixel intensity distribution, and multi-modal acquisition of adjacent tissue cross-sections) means that registration of digital microscopy images, in particular of multi-modal images, for whole slide microscopy is not trivial and again requires intensive computation which limits the performance of the digital pathology systems in terms of speed.
Again, even though the two solutions described above (hardware or image processing algorithm optimization, and down sampling the whole slide image) may be used to improve the performance of the digital pathology system, this may be not be sufficient.
Therefore, the invention desires to solve the above-mentioned problems.